Datacenter Storage Architecture for MapReduce Applications
نویسندگان
چکیده
Data-intensive computing systems running MapReducestyle applications are currently architected with storage local to computation in the same physical box. This poster argues that upcoming advances in converged datacenter networks will allow MapReduce applications to utilize and benefit from network-attached storage. This is made possible by properties of all MapReduce-style applications, such as streaming storage access patterns. By decoupling computation and storage, stateless compute nodes containing inexpensive, low-power processors can be deployed in large numbers to increase application performance, improve reliability, and decrease power consumption.
منابع مشابه
DCR: Replay-Debugging for the Datacenter
We’ve built a tool for debugging non-deterministic failures in production datacenter applications. Our system, called DCR, is the first to efficiently record and replay large scale, distributed, and data-intensive systems such as HDFS/GFS, HBase/Bigtable, and Hadoop/MapReduce. The enabling idea behind DCR is that debugging doesn’t require a precise replica of the original datacenter run. Instea...
متن کاملThe Datacenter Needs an Operating System
Clusters of commodity servers have become a major computing platform, powering not only some of today’s most popular consumer applications—Internet services such as search and social networks—but also a growing number of scientific and enterprise workloads [2]. This rise in cluster computing has even led some to declare that “the datacenter is the new computer” [16, 24]. However, the tools for ...
متن کاملTime-aware Software Defined Networking for OpenFlow-based Datacenter Optical Networks
Data center networks are considered to make use of the computing and storage resources in data centers, which include intra-datacenter and inter-datacenter networks. Both of them will depend on the optical networking due to its advantages, such as low latency, high bandwidth, and low energy consumption. Data center interconnected by flexi-grid optical networks is a promising scenario to allocat...
متن کاملTowards Energy Efficient MapReduce
Energy considerations are important for Internet datacenters operators, and MapReduce is a common Internet datacenter application. In this work, we use the energy efficiency of MapReduce as a new perspective for increasing Internet datacenter productivity. We offer a framework to analyze software energy efficiency in general, and MapReduce energy efficiency in particular. We characterize the pe...
متن کاملImproved routing algorithms in the dual-port datacenter networks HCN and BCN
We present significantly improved one-to-one routing algorithms in the datacenter networks HCN and BCN in that our routing algorithms result in much shorter paths when compared with existing routing algorithms. We also present a much tighter analysis of HCN and BCN by observing that there is a very close relationship between the datacenter networks HCN and the interconnection networks known as ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2009